Q-Learnheuristics: towards data-driven balanced metaheuristics

One of the central issues that must be resolved for a metaheuristic optimization process to work well is the dilemma of the balance between exploration and exploitation. The metaheuristics (MH) that achieved this balance can be called balanced MH, where a Q-Learning (QL) integration framework was proposed for the selection of metaheuristic operators conducive to this balance, particularly the selection of binarization schemes when a continuous metaheuristic solves binary combinatorial problems. In this work the use of this framework is extended to other recent metaheuristics, demonstrating that the integration of QL in the selection of operators improves the exploration-exploitation balance. Specifically, the Whale Optimization Algorithm and the Sine-Cosine Algorithm are tested by solving the Set Covering Problem, showing statistical improvements in this balance and in the quality of the solutions

A Binary Black Widow Optimization Algorithm for Addressing the Cell Formation Problem Involving Alternative Routes and Machine Reliability

The Cell Formation Problem (CFP) involves the clustering of machines to enhance productivity and capitalize on various benefits. This study addresses a variant of the problem where alternative routes and machine reliability are included, which we call a Generalized Cell Formation Problem with Machine Reliability (GCFP-MR). This problem is known to be NP-Hard, and finding efficient solutions is of utmost importance. Metaheuristics have been recognized as effective optimization techniques due to their adaptability and ability to generate high-quality solutions in a short time. Since BWO was originally designed for continuous optimization problems, its adaptation involves binarization. Accordingly, our proposal focuses on adapting the Black Widow Optimization (BWO) metaheuristic to tackle GCFP-MR, leading to a new approach named Binary Black Widow Optimization (B-BWO). We compare our proposal in two ways. Firstly, it is benchmarked against a previous Clonal Selection Algorithm approach. Secondly, we evaluate B-BWO with various parameter configurations. The experimental results indicate that the best configuration of parameters includes a population size (Pop) set to 100, and the number of iterations (Maxiter) defined as 75. Procreating Rate (PR) is set at 0.8, Cannibalism Rate (CR) is set at 0.4, and the Mutation Rate (PM) is also set at 0.4. Significantly, the proposed B-BWO outperforms the state-of-the-art literature’s best result, achieving a noteworthy improvement of 1.40%. This finding reveals the efficacy of B-BWO in solving GCFP-MR and its potential to produce superior solutions compared to alternative methods

Binarization of Metaheuristics: Is the Transfer Function Really Important?

In this work, an approach is proposed to solve binary combinatorial problems using continuous metaheuristics. It focuses on the importance of binarization in the optimization process, as it can have a significant impact on the performance of the algorithm. Different binarization schemes are presented and a set of actions, which combine different transfer functions and binarization rules, under a selector based on reinforcement learning is proposed. The experimental results show that the binarization rules have a greater impact than transfer functions on the performance of the algorithms and that some sets of actions are statistically better than others. In particular, it was found that sets that incorporate the elite or elite roulette binarization rule are the best. Furthermore, exploration and exploitation were analyzed through percentage graphs and a statistical test was performed to determine the best set of actions. Overall, this work provides a practical approach for the selection of binarization schemes in binary combinatorial problems and offers guidance for future research in this field

Swarm-Inspired Computing to Solve Binary Optimization Problems: A Backward Q-Learning Binarization Scheme Selector

In recent years, continuous metaheuristics have been a trend in solving binary-based combinatorial problems due to their good results. However, to use this type of metaheuristics, it is necessary to adapt them to work in binary environments, and in general, this adaptation is not trivial. The method proposed in this work evaluates the use of reinforcement learning techniques in the binarization process. Specifically, the backward Q-learning technique is explored to choose binarization schemes intelligently. This allows any continuous metaheuristic to be adapted to binary environments. The illustrated results are competitive, thus providing a novel option to address different complex problems in the industry

Continuous Metaheuristics for Binary Optimization Problems: An Updated Systematic Literature Review

For years, extensive research has been in the binarization of continuous metaheuristics for solving binary-domain combinatorial problems. This paper is a continuation of a previous review and seeks to draw a comprehensive picture of the various ways to binarize this type of metaheuristics; the study uses a standard systematic review consisting of the analysis of 512 publications from 2017 to January 2022 (5 years). The work will provide a theoretical foundation for novice researchers tackling combinatorial optimization using metaheuristic algorithms and for expert researchers analyzing the binarization mechanism’s impact on the metaheuristic algorithms’ performance. Structuring this information allows for improving the results of metaheuristics and broadening the spectrum of binary problems to be solved. We can conclude from this study that there is no single general technique capable of efficient binarization; instead, there are multiple forms with different performances

A Novel Learning-Based Binarization Scheme Selector for Swarm Algorithms Solving Combinatorial Problems

Currently, industry is undergoing an exponential increase in binary-based combinatorial problems. In this regard, metaheuristics have been a common trend in the field in order to design approaches to successfully solve them. Thus, a well-known strategy includes the employment of continuous swarm-based algorithms transformed to perform in binary environments. In this work, we propose a hybrid approach that contains discrete smartly adapted population-based strategies to efficiently tackle binary-based problems. The proposed approach employs a reinforcement learning technique, known as SARSA (State–Action–Reward–State–Action), in order to utilize knowledge based on the run time. In order to test the viability and competitiveness of our proposal, we compare discrete state-of-the-art algorithms smartly assisted by SARSA. Finally, we illustrate interesting results where the proposed hybrid outperforms other approaches, thus, providing a novel option to tackle these types of problems in industry

Analysis and Prediction of Engineering Student Behavior and Their Relation to Academic Performance Using Data Analytics Techniques

This study focuses on identifying personality traits in computer science students and determining whether they are related to academic performance. In addition, the importance of the personality traits based on motivation scale and depression, anxiety, and stress scales were measured. A sample of 188 students from the Computer Engineering Schools of the Pontifical Catholic University of Valparaíso was used. Through econometric two-stage least squares and paired sample correlation analysis, the results obtained indicate that there is a relation between academic performance and the personality traits measured by educational motivation scale and the ranking of university entrance and gender. In addition, these results led to characterization of students based on their personality traits and provided elements that may enhance the development of an effective personality that allows students to successfully face their environment, playing an important role in the educational process